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Student Evaluations of Teaching (SETs) are used by nearly all public and private universities as one 
means to evaluate teaching effectiveness. A majority of these universities have transitioned from the 
traditional paper-based evaluations to online evaluations, resulting in a decline in overall response 
rates. This has led to scepticism about the validity and reliability of the SETs. In this study, a large, 
US public university transitioned to online SETs in 2007 and suffered a decline in overall response 
rates from 73% for the paper-based evaluations in 2006 to a low of 43%. The aim of this study was 
to determine successful strategies used by instructors to improve their own SET response rates. A 
survey was conducted of faculty members who had high response rates, and the data were analyzed 
to determine which strategies were being employed. The study found that when instructors show 
students they care about evaluations, response rates tend to be higher. The results from the study 
have been turned into a FAQ on myths and suggestions that has been distributed to the faculty at the 
university to provide guidelines for increasing response rates on SETs. 


Universities are facing increasing pressure to 
assess educational outcomes. In this climate, one 
concrete way to assess teaching effectiveness is through 
end of course evaluations. Although several studies 
have shown student evaluations to be reliable and 
somewhat valid, end of course evaluations are not 
without their problems (Aleamoni, 1999; Centra, 2003; 
Hobson & Talbot, 2001). Individual faculty members 
are often concerned with the validity, reliability, and 
usefulness of the SETs in assessing their individual 
teaching effectiveness. Owing to small sample sizes, 
the data obtained from these evaluations can lack 
statistical significance, and results can be biased. 
Especially when response rates are low, instructors are 
concerned that only dissatisfied or less successful 
students respond to SETs. Research refutes this 
common myth, as more successful and engaged 
students tend to complete online evaluations (Adams & 
Umbach, 2012). Obtaining a high response rate can 
help alleviate some of these concerns. Since the 
majority of institutions use SETs to inform decisions 
about faculty salaries as well as reappointment, 
promotion, and tenure, ensuring statistically significant 
data through high response rates is a goal shared by 
administrators and faculty alike (Education Advisory 
Board, 2009; Haskell, 1997). For example, one study 
showed that instructors with class sizes under 10 should 
have at least a 75% response rate under liberal (10% 
sampling error) conditions to create reliable feedback 
and 100% under stringent (3% sampling error) 
conditions (Nulty, 2008), while others refute this 
notion, noting that response rates under 100% are not 
satisfactory as they may not be generalizable to the 
entire class, especially for small class sizes (Kulik, 
2009). Despite the importance of obtaining a high SET 
response rate, research on best practices in increasing 
evaluation response rates is relatively scarce (Misra, 


Stokols, & Marino, 2013), and there have been calls by 
researchers for more study on strategies for increasing 
response rates (Adams & Umbach, 2012; Goodman, 
Anson, & Belcheir, 2015). 

According to the University Planning and Analysis 
(UPA) and the Evaluation of Teaching Committee 
(EOTC) at the university under study, response rates for 
end-of-course evaluations have been gradually declining 
since the instrument began being administered online. The 
EOTC was considering recommending changes to the 
current “no-incentives” policy by allowing incentives for 
students who complete SETs as a potential way to boost 
response rates. The EOTC knew different strategies were 
being used by instructors to help increase response rates, 
but it was not known which strategies were being 
employed, which strategies worked, and which strategies 
aligned with current university policy. Misra et al. (2013) 
found, “Developing effective strategies for increasing 
response rates can help reduce nonresponse biases in 
survey data and improve the quality of research findings” 
(89). The purpose of this study was to determine which 
strategies were being used by faculty members to 
effectively increase SET response rates. 

Review of the Literature 

SETs are often the primary assessment of teaching 
performance in institutions of higher education in the 
U.S. (Pounder, 2007), but as with all types of evaluation, 
they are inherently political (Russ-Eft & Preskill, 2009). 
Student evaluation of teaching in higher education was 
initially intended to help instmctors improve their 
teaching and/or student learning. It was only later that 
the results were commonly used for promotion and 
tenure purposes (Lindahl & Unger, 2010). Marzano 
(2012) reported that teachers perceived evaluation in one 
of two ways: for measurement and for development. 



Chapman and Jones 


Online End-of-Course Evaluations 48 



Most SETs used in higher education today are for 
the purposes of measurement and not for development 
and are typically summative since they are performed at 
the end of the semester. 

Instructors have long argued the problems with 
SETs, mainly because of their use for promotion and 
tenure purposes. Critiques abound on the usefulness, 
validity, and reliability of these traditionally end-of- 
term instructor evaluations. Lindahl and Unger (2010) 
claimed that the situation itself leads to atypical 
behavior: “The structure of the collection process itself, 
involving a group situation, heightened emotional 
arousal, and anonymity, encourages deindividuation 
and may allow the mechanisms of moral disengagement 
to operate, permitting behavior that students would 
never engage in face-to-face” (73). 

There are additional reasons why end-of-course 
evaluations at research-intensive universities rarely result 
in instructional improvement. SETs habitually get 
distilled down to a single quantitative number whether 
high or low; they often tell one nothing about how to 
improve teaching, and often ratings are based on a 
consumerism model that is focused on entertainment 
level or difficulty of the course (Wright, 2000). Courses 
vary widely by discipline, class size, student 
demographics, and outcomes, but end-of-course 
evaluations are usually standardized and may not be 
suitable across institutions (Richardson, 2005). 
McCullough and Radson (2011) suggested that SETs are 
often not calculated correctly because they are based on 
ordinal data but analyzed as interval data. Add to this the 
issue that students are not trained to rate any one 
question in the same way. This leads to unreliable and 
likely invalid results. When the stakes are high, the 
pressure to make false or misleading statements 
increases. Studies have shown that students lie on 
faculty evaluations, especially in cases where the student 


has an axe to grind (Clayson, 2008). However, some 
studies show that the dissatisfied or poorer students are 
less likely to fill out the SETs (Adams & Umbach, 2012; 
Avery, Bryant, Mathios, Kang, & Bell, 2006; Fidelman, 
2007; Sax, Gilmartin, & Bryant, 2003). Adams and 
Umbach (2012) found students who have spent time 
working to get a good grade are more likely to fill out 
SETS and surmised that students with higher GPAs and 
course grades have “the intellectual ability to evaluate 
the course at a meaningful level” (586). 

Online SETs and Response Rates 

To complicate matters more, most SETs are now 
administered online. Potential advantages of performing 
SETs online include standardization across the institution, 
no loss of class time to perform SETs, reductions in cost 
due to the absence of printing, distributing, and collecting 
results (Bothell & Henderson, 2003), getting feedback to 
instructors more efficiently, and reduction of errors for 
partially or improperly filled out forms. Online SETs can 
be argued to have more flexibility in the time and location 
for completion (Cummings, Ballantyne, & Fowler, 2001), 
which allows students to write more thoughtful comments 
online than on paper (Adams & Umbach, 2012; 
Ballantyne, 2003; Cummings et al., 2001; Hativa, 2013; 
Kasiar, Schroeder, & Holstad, 2002; Stowell, Addison, & 
Smith, 2012). In addition to the cost savings, Dommeyer, 
Baum, Hanna, & Chapman (2004) pointed out that online 
evaluations may help minimize the faculty influence over 
in-class SETs (e.g., activities that happen prior to 
evaluation, presence of the faculty, and peer influence) as 
well as allow more students to complete them (i.e., if they 
were absent on the day of the in-class evaluation.) Online 
administration provides for more anonymity, eliminating 
potential handwriting recognition of paper-based SETs 
(Avery et ah, 2006). 
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Multiple studies reported that while response rates 
for online SETs initially average near 60%, they soon 
drop off to the 30 to 40 percentile range (Avery et al. 
2006; Nulty, 2008; Sax et al., 2003). As seen in Figure 
1, this phenomenon occurred at the university under 
study when it moved to online evaluations in the spring 
of 2007, reaching a low of 43% in the fall 2011 and 
spring 2012 semesters. While these levels of response 
rate may hold some statistical significance in large 
courses, smaller classes are more problematic as 40% 
of a class of 20 is only eight responses (see Table 1 for 
recommended levels for validity.) SETs with low 
response rates may not be representative of the whole 
and add to the argument against making instructional 
changes or personnel decisions based upon such 
feedback, although one study found that scoring 
methods were similar for both forms of administration 
(Fike, Doyle, & Connolly, 2010). 

Low response rates for online SETs are partially 
due to a lack of motivation for filling them out since 
students are no longer in class. Students do not 
necessarily benefit from SETs (Bullock, 2003) as they 
are done at the end of the term, and thus can provide 
only a snapshot of the instructional process at a point 
when the current students will not experience 
instructional improvements. Students perceive that 
evaluations have no effect on an instructor’s teaching 
effectiveness or performance review. Often they are 
left with the notion that no one but the individual 
instructors will see them or that the SET results are not 
taken seriously (Spencer & Schmelkin, 2002). These 
perceptions have some validity as research has shown 
that faculty do not view student evaluations as valuable 
for improving instruction and report not making 
changes based SETs (Beran & Rokosh, 2009; Gaillard, 


Mitchell, & Kavota, 2006). SETs are fraught with 
problems, and although only a sampling of the 
criticisms is presented here, the literature is clear that 
the low and declining response rates for online SETs 
present fundamental problems as well as 
misperceptions (Avery et al., 2006, Dommeyer et al., 
2004; Norris & Conn, 2005; Nowell, Gale, & Handley, 
2010; Stowell et al., 2012). 

Response/Non-response Rates 

Low response rates for online SETs are a 
recognized problem in higher education and have been 
studied from a variety of perspectives. This problem 
stems from the concern that low response rates have the 
potential to create bias if the students filling out the 
evaluations are not representative of the entire class 
population. Adams and Umbach (2012) found that that 
non-response bias may actually double-bias SET results 
as “not only are students with higher grades typically 
awarding higher ratings, but they are also the ones who 
are more likely to respond” (586). They also found that 
engaged students were more likely to respond to 
courses in their major, but the more SET requests sent 
to a student, the more unlikely the student is to respond 
(i.e., survey fatigue). It is no surprise that in an earlier 
study some instructors were found to prefer the 
traditional paper method because of their beliefs that 
they can achieve higher response rates and a more 
accurate representation of the population (Dommeyer et 
al., 2004). But, as mentioned earlier, in-class 
evaluations are not without their own issues (e.g., 
potential instructor and/or peer influence, students 
filling out multiple evaluations, concern of student 
anonymity, etc.) 


Table 1 


Suggested Minimum Response Rates Required for Validity of Data (Adapted from Nulty, 2008) 


Class Size 

Recommended Rates under 
Liberal Conditions* 

Recommended Rates under Stringent 
Conditions** 

10 

75% 

100% 

30 

48% 

96% 

50 

35% 

93% 

70 

28% 

91% 

100 

21% 

87% 

200 

12% 

77% 

300 

8% 

70% 

500 

5% 

58% 


*10% sampling error; 80% confidence level; **3% sampling error; 95% confidence level 
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Incentives and Increasing Response Rates 

Misra, Stokols, and Marino (2011) found that 
social norm-based appeals for issues such as social 
cooperation and social responsibility were effective 
in increasing web-based response rates. A number 
of researchers have noted that reminding students 
about the evaluations as well as letting the students 
know the importance of SETs has helped response 
rates rise (Dommeyer et al., 2004; Goodman et ah, 
2015; Johnson 2002; Laubsch 2006; Nulty, 2008; 
University of British Columbia, 2010). 
Additionally, researchers have shown that 
instructors who performed a formative mid- 
semester evaluation as part of their class gained 
between 9% and 16% in response rates (Crews & 
Curtis, 2011; Lewis, 2001b; McGowen & 
Osgathorpe, 2011; Tucker, Jones, & Straker, 2008). 
Students respond positively when they feel their 
comments will make a difference in improving a 
class. Students then become more engaged in the 
course as well as better evaluators (Lewis, 2001b). 
They are more motivated if they feel their voices 
will be heard and it can begin with simply stating 
how SETs results are used in the course syllabus 
(Chen & Hoshower, 2003; Tucker et ah, 2008). 

Several studies have examined aspects of the use of 
incentives to increase response rates in online surveys 
(Crews & Curtis, 2011; Dommeyer et ah, 2004; 
Goodman et ah, 2015; McGourty, Scoles, & Thorpe, 
2002a, b). Cook, Heath, and Thompson (2000) found 
that personalized correspondence is linked to higher 
response rates in electronic surveys. Students are also 
more likely to reply to surveys they find more relevant. 
One study found that the best determinant of response 
rate was issue salience. In other words, the more salient 
the issue to the respondent, the more likely he or she is 
to respond (Sheehan & McMillan, 1999). Interestingly, 
Cook and colleagues (2000) found that the use of 
incentives was negatively associated with response 
rates and resulted in more homogeneous responses. 
Several researchers have discussed the importance of 
giving positive incentives such as extra credit or bonus 
points in order to achieve high response rates 
(Anderson, Cain, & Bird, 2005; Goodman et al., 2015) 
or making SET completion an assignment for the class 
(Ravenescroft & Enyeart, 2009). Another study found 
that entering students into a random drawing for a cash 
prize upon completing their evaluations worked as an 
incentive option but was not highly effective 
(Ballantyne, 2003). Some universities withhold early 
access to grades unless the evaluations are filled out 
(Anderson et al., 2005). Clearly, the research on which 
incentives work to increase response rates in web-based 
evaluations is mixed (Misra et al., 2011). 


Methodology 

Because of poor response rates for SETs (see 
Figure 1), the EOTC at the university under study 
wanted to know what could be done to improve them. 
This study was designed to determine the following: 

What strategies are instructors using to 
successfully improve response rates in SETs? 

How do these strategies compare to the university 
policy? 

What strategies should be recommended for use 
throughout the university? 

The university under study is a large (over 33,000 
students) research intensive institution located in the 
United States. SET process and procedure is governed 
by policy and administered by a centralized division 
reporting to the university’s Provost. Prior to spring of 
2007, when they began to be administered online, SETs 
were administered in a face-to-face format. Since that 
time, response rates have steadily declined. 

The University’s SET is administered online 
through a proprietary system and includes 12 Likert 
scale questions and three open-ended questions to allow 
for comments. Deans, department heads, and 
instructors may add a limited number of their own 
questions to this set of 15 common-core questions. The 
system automatically sends out generic email reminders 
several times to those students who have not filled out 
their evaluations. Instructors cannot see their SET 
results until after the last official day to post final 
grades but can monitor the response rates online and in 
real time (NCSU, 2013, para. 1). 

The policies relating to strategies and/or incentives 
for completion of SETs are clear and cover such topics 
as the instrument, the scope, and the procedures. 
Specifically, students are not required to fill out the 
evaluation (NCSU, n.d., para. 31) and incentives to 
increase response rate are forbidden (para. 33). 

Population and Data Collection 

The population under study consisted of 205 
instructors (out of approximately 950 total faculty 
members who taught at least one course in the previous 
semester) that received an SET response rate of 70% or 
higher. Because the objective of the study was to find 
successful strategies for increasing response rate, the 
decision was to limit participants to only those who 
taught at least one course in the semester that had a 
70% or higher response rate. Seventy percent was 
selected in order to find successful strategies, and 70% 
covered most requirements for survey validity for class 
sizes over ten in liberal conditions (see Table 1). The 
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Table 2 

List of Survey Strategies to Increase Response Rates 

_ Strategy _ 

Sent announcements through Moodle asking students to complete evaluations. If so, how many announcements do 
you generally send? 

Sent personal e-mails to students asking them to complete evaluations. If so, how many emails do you generally 
send? 

Talked about the importance of ClassEval in my class. 

Included statements on the syllabus about ClassEval and its importance in my class. 

Worked to create a climate in my class that reflects mutual respect between instructor and students. 

Held my course in (or took my class to) a computer lab and allowed time for students to complete the evaluation 
while a moderator was there. 

Encouraged students to bring laptops/tablets/smartphones to class and allowed time for students to complete the 
evaluation while a moderator was there. 

Told my students how I use student evaluation feedback to modify my course. 

Offered a mid-semester evaluation where students could give feedback and then used that feedback to modify my 
course. 

Forwarded an e-mail from a Department Head or Dean about the importance of course evaluations to my College 
or Department. 

Offered to bring snacks to class or final if a particular response rate was achieved. 

Added bonus points to students' test or assignments if certain course response rates were achieved. 

Dropped a low assignment grade for all students if certain response rates were achieved. 

Increased all students’ grades if certain course response rates were achieved. 

Added a bonus/extra credit question or questions to the final if a certain course response rate was achieved. 

No actions were taken to increase ClassEval response rates in these courses. _ 


survey was anonymous, was open for three weeks, and 
used two follow-up reminders. Out of the population of 
205, 120 participants completed the survey resulting in 
a response rate of 59%. 

Instrumentation 

A Web-based survey instrument was developed 
that listed 15 different strategies (see Table 2) that 
were either found in the literature as having been 
associated with higher response rates for SETs or 
that members of the EOTC heard were being used. 
The list was reviewed for face validity by members 
of the committee. It should be noted that the SET 
instrument used at this institution is called 
ClassEval. In addition, there were two text boxes in 
which respondents could add alternative methods 
that were not represented in the list. The survey 
began with qualifying questions (see Table 3) that if 
answered in a particular matter would disqualify a 
participant. This was done to assure that 


participants actually did teach at least one course in 
the term that received a 70% or higher response 
rate. Because of the university policy against 
incentives, the study did not collect any identifying 
characteristics which could be linked back to a 
particular respondent, class, and/or set of 
evaluations. 

The final part of the survey listed the 15 potential 
strategies along with two spaces for respondents to add 
strategies not represented as seen in Table 2. The prompt 
was stated: “In those courses that received a response 
rate of 70% or higher, select all of the ways you or 
someone else took action to increase response rates.” 

Findings 

The instrument included three demographic 
questions. The first question asked respondents to 
report the number of course sections they taught in the 
prior semester (see Figure 2). Those that reported they 
taught five or more sections were likely considering 
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Table 3 

Qualifying Questions 

Qualifying Question _ How Participant was Disqualified _ 

How many course sections did you teach in Fall 2012? Participant disqualified if response was zero 

Of these courses, how many of them received an end of Participant disqualified if response was zero 
course evaluation response rate of 70% or higher (an 
estimate is fine)? 


Figure 2. 

Number of Sections Taught in Semester. 



labs, independent studies, and other course structures 
that differ from the standard three credit hour course. 
The second question asked respondents to specify the 
number of sections received an SET response rate of 
70% or greater as seen in Figure 3. 

The third question asked respondents to estimate 
the number of students in the class for those with an 
SET response rate of 70% or greater. Here the majority 
of the classes had from 11 to 25 students enrolled (see 
Figure 4). 

Strategies 

Respondents were asked to select strategies used to 
increase SET response rate in their courses that had a 70% 
or greater response rates in the previous semester. They 
could select from the list of 15 options in Table 2, or they 
could add additional strategies. They were allowed to 
select more than one option. The list of strategies included 
those that are considered incentives against the current 
policy as well as non-incentive strategies. Figure 5 shows 
the results of comparing instructors based on their use of 
incentives. As seen, the number of faculty using no form 


of incentive strategies is statistically higher than those that 
used any form of incentive. 

Response frequencies for each strategy are listed in 
Table 4. The most used strategies seen in Table 4 are 
not ones associated with giving away bonus points or 
altering assignments, but with the way in which the 
instructors approached students about the SET process. 
The most often used strategy was merely talking about 
the importance of SETs in their classes, followed 
closely by creating an environment of mutual respect in 
the classroom. The assumption here is that mutual 
respect creates an environment where students want to 
fill out evaluations. The third most commonly used 
strategy (and the only other strategy used by more than 
half of the respondents) was instructors who told their 
students how they used evaluation results to modify 
their courses. The next three most highly rated 
strategies were used by 27% to 35% of the respondents 
and were all related to the ways in which information 
about the SET was communicated. 

During analysis, incentives were also categorized 
by type of incentive, a category that classified the 
strategy as either “No Incentive,” a “Red Incentive,” or 
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Figure 3 

Number of Sections with SET Response Rate of 70% or Higher. 
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Figure 4. 

Number of Students in Sections with SET Response Rates 70% or Higher. 
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Figure 5 

Proportion Testing of Faculty Using any Form of Incentive (95% confidence error bars) 
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Table 4 


Response Frequencies for Strategies to Increase Response Rates 


Strategy 

N 

% 

Type of Incentive 

1 

Talked about the importance of ClassEval in my class. 

97 

87% 

No Incentive 

2 

Worked to create a climate in my class that reflects mutual 
respect between instructor and students. 

93 

83% 

No Incentive 

3 

Told my students how I use student evaluation feedback to 
modify my course. 

87 

78% 

No Incentive 

4 

Sent announcements through Moodle asking students to 
complete evaluations. If so, how many announcements do you 
generally send? 

39 

35% 

No Incentive 

5 

Sent personal e-mails to students asking them to complete 
evaluations. If so, how many emails do you generally send? 

36 

32% 

No Incentive 

6 

Included statements on the syllabus about ClassEval and its 
importance in my class. 

30 

27% 

No Incentive 

7 

Encouraged students to bring laptops/tablets/ smartphones to 
class and allowed time for students to complete the evaluation 
while a moderator was there 

26 

23% 

No Incentive 

8 

Offered a mid-semester evaluation where students could give 
feedback and then used that feedback to modify my course. 

25 

22% 

No Incentive 

9 

Added bonus points to students' test or assignments if certain 
course response rates were achieved. 

15 

13% 

Red Incentive 

10 

Held my course in (or took my class to) a computer lab and 
allowed time for students to complete the evaluation while 
moderator was there. 

11 

10% 

No Incentive 

11 

Increased all students’ grades if certain course response rates 
were achieved. 

8 

7% 

Red Incentive 

12 

Added a bonus/extra credit question or questions to the final if a 
certain course response rate was achieved. 

8 

7% 

Grey Incentive 

13 

Dropped a low assignment grade for all students if certain 
response rates were achieved. 

4 

4% 

Red Incentive 

14 

Forwarded an e-mail from a Department Head or Dean about the 
importance of course evaluations to my College or Department. 

2 

2% 

No Incentive 

15 

Offered to bring snacks to class or final if a particular response 
rate was achieved. 

2 

2% 

Grey Incentive 

16 

No actions were taken to increase ClassEval response rates in 
these courses. 

0 

0% 

No Incentive 


*Respondents could choose more than one strategy. 


a “Grey Incentive.” These categories were defined by 
the EOTC whereas a red incentive was classified as 
being totally against policy while grey incentive 
strategies were against the policy, but not as egregious 
because students were considered to be affected in the 
same manner. Both types of strategies were considered 
incentives currently prohibited by university policy. 
This categorization is displayed in Table 4. 

The issue of grade influence only begins to show at the 
ninth most often used strategy where instructors added 
bonus points to tests or assignments if a certain response rate 
was achieved (13%), and strategies ranked at 11, 12 and 13 
also refer to strategies that could will likely influence grades. 

The total number of non-incentive strategies 
employed by faculty who used at least one incentive 


versus those faculty who did not was statistically the same, 
as seen in Figure 6. Also, most instructors who received 
high response rates employed an average 4.3 different 
strategies. Even when a faculty member used a prohibited 
incentive to increase their response rates, he or she still 
employed an average of 4.5 non-prohibited strategies. 
Because the group distributions of “No Incentive” and 
“Incentive” were not normally distributed, a 
Wilcoxon/Kmska-Wallis Test using JMP™ software was 
employed to test the null hypothesis that the samples come 
from the same distribution. Since the p-value is 0.61, the 
null hypothesis cannot be rejected and it can be concluded 
that the number of non-incentive strategies employed by 
faculty who use at least one incentive is the same as 
faculty who do not employ incentives. 
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In Figure 5 it was shown that statistically, more 
instructors are employing incentives in alignment 
with university policy as compared to those that are 
using strategies prohibited by policy. However, 
class size seems to impact those decisions. Figure 7 
shows the contingency analysis when doing the 
same comparison with regard to class size: small (5 
to 25), medium (26 to 75), and large (greater than 
75). The class sizes from Figure 4 had to be 


merged to ensure at least five items of each class 
size occurred for each category (i.e., no incentive 
and incentive) to make the analysis valid. The null 
hypothesis (the proportion of faculty employing 
incentives for all three class sizes is the same) is 
rejected because the p-value for the Chi square test 
that is less than 0.0001. The larger the class size, 
the more likely a faculty member was to use a 
prohibited incentive to help increase response rates. 


Figure 6 

Comparing the Number Non-Incentive Strategies Employed by Each Respondent. 
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Figure 7 

Comparing the Use of Incentives Based on Class Size. 
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Additional Strategies 

Thirty respondents submitted strategies they felt 
were not represented in Table 4, but after closer 
inspection, only 10 were considered additional 
strategies (Table 5). The first strategy was related to 
evoking student responsibility where instructors would 
not only talk about the SET in class, but would also 
imply the student had a social responsibility in helping 
to create better learning environments and providing 
input affecting the career of the instructor. Some 
instructors told students that evaluation was a privilege 
that was fought for decades ago and others described it 
as a responsibility. This can clearly be seen in one 
instructor’s comment "‘I emphasize that I worked hard 
to deliver their course and if they respect that fact, I am 
entitled to feedback- positive or negative.” 

Another instructor described her strategy, “I 
explain that low response rates mean that the evals, 
whether positive or negative, are somewhat suspect.” 
The next most often mentioned additional strategy was 
giving students time off: “1 let students leave early or 
not have class if a certain response rate was achieved,” 
explained an instructor. Note that all mention of time 
off related to the last day of class, whether it was part of 
the day or the entire day. 

Discussion 

This study sought to determine the types of 
strategies that are successfi.il in increasing response 
rates to SETs. Although the findings are limited due to 
the self-reporting nature of the study, there are still 
valuable findings and implications for policy, 
instructors, and administrators. While instructors can 
employ a myriad of methods, three strategies were used 
by more than 75% of the respondents in this study. 
These strategies were: 

1. Talked about the importance of class 
evaluations in my class; 

2. Worked to create a climate in my class that 
reflects mutual respect between instructor and 
students; and 

3. Told my students how I use student evaluation 
feedback to modify my course. 

These results clearly show that at this institution, 
high SET response rates are more associated with 
course climate and instructor-student communication 
than with incentives given to students. In fact, the top 
eight strategies did not include incentives and it was 
only at 13% when actual incentives appear in the results 
(adding points to tests or assignments.) This 
contradicts the findings of Goodman and colleagues 


(2015), who determined that grade incentives were the 
most effective way of increasing response rates. 

Policy and Standards 

When reviewing the usage of strategies that are 
acceptable to the institution and incentives that are not, 
the results have clear policy implications. The policy at 
the university under study states, “There is no penalty to 
students who decline to submit evaluations,” and, “No 
form of incentive should be provided to increase 
response rate.” While the great majority of instructors 
achieving a 70% or higher response rate used strategies 
that would not be considered incentives, there were 
instructors using incentives that are opposed to the 
institutional policy. Table 6 displays the strategies 
instructors used that maybe considered incentive-based. 

Implications for Instructors 

The clearest implication from this study for 
instructors is to talk about student evaluations of 
teaching with their students. This not only includes 
explaining their purposes, but also focusing on how the 
instructor uses the information and who benefits from 
the information that is submitted via an SET (Lewis, 
2001a). Results of this study support the case for 
creating a climate of mutual respect, one where student 
opinions are respected and addressed and instructor 
needs are taken into consideration. This can be 
accomplished through class discussion and by modeling 
behaviors such as using formative evaluations of 
teaching and pointing out to students the changes that 
result from analysis of the data. The key information 
here is that incentives are not only against policy, likely 
to bias data, and have questionable ethical implications, 
but they also do not work as well as simply reinforcing 
the importance of participating in the process making 
students feel their voices make a difference. 

Implications for Policy and Administrators 

The results of this study in no way support the use 
of incentives to raise SET responses rates. Policy 
makers should focus on rules and processes that enable 
faculty members to conduct productive evaluation 
discussions in all classes. Steps should be taken to 
reduce the conflict between the use of SET results for 
course improvement and the use for promotion and 
tenure purposes. When an institution places high 
importance on SET data for promotion and tenure, it 
may also increase the likelihood of an instructor to use 
incentives to increase response rates. Should SETs be 
primarily used to improve instruction, response rates 
and validity become less of a high-stakes issue and the 
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Table 5 


Additional Strategies via Open-ended Responses 


Strategy 

N 

% 

1 

Evoked Student Responsibility or Guilt 

4 

4% 

2 

Make Learning about Statistical Significance a Part of Class Content 

4 

4% 

3 

Gave Students Time Off 

3 

3% 

4 

Gave Bonus attached to Honesty Attestation 

2 

2% 

5 

Commanded Students to Complete Evaluation 

1 

1% 

6 

Appealed from the Student Perspective 

1 

1% 

7 

Withheld Final Grades 

1 

1% 

8 

Created Competition Among Sections 

1 

1% 

9 

Altered Final Exam 

1 

1% 

10 

Withheld Study Aids 

1 

1% 



Table 6 

Strategies that May Be Construed as Incentives 



Strategy 

N 

% 

1 

Increased all students’ grades if certain course response rates were achieved. 

15 

13% 

2 

Added a bonus/extra credit question or questions to the final if a certain course 
response rate was achieved. 

11 

10% 

3 

Dropped a low assignment grade for all students if certain response rates were 
achieved. 

8 

7% 

4 

Gave Bonus attached to Honesty Attestation 

2 

2% 

5 

Offered to bring snacks to class or final if a particular response rate was achieved. 

2 

2% 

6 

Withheld Final Grades 

1 

1% 

7 

Altered Final Exam 

1 

1% 

8 

Withheld Study Aids 

1 

1% 


pressure to increase response rate somewhat 
diminishes. The goal for policy makers should be to 
reduce the impetus for participating in activities that 
would bias results or be considered unethical. As 
echoed by the American Evaluation Association 
(AEA) evaluation standards (AEA, 2015), SET 
policy should project and guard against unintended 
consequences, such as extreme urgency in inflating 
SET response rates, as well as avoid conflicts of 
interest between the formative and summative uses 
of the SET. In order for SETs to be valid and 
reliable, policy makers should decide their primary 
purpose (i.e., course improvement or faculty 
promotion and tenure). 

Conclusion 

This study examined practices among 
instructors who had high SET response rates in 
order to determine best practices in increasing end 
of course evaluation response rates. Findings 
indicated that the most common strategies to 
successfully increase SET response rates were: 


a. Discussing the importance of evaluation feedback 
and how it will be used to inform future courses 

b. Working to create a classroom culture that reflects 
mutual respect between instructor and students. 

Showing students “that their input is important in the 
collaborative venture of teaching and learning” is mutually 
beneficial to instructor and student (Keutzer, 1993, p. 240). 
Use of incentives was not employed as widely as the 
investigators expected. Based on the results, an FAQ 
document was created to assist faculty in increasing 
response rates without the use of incentives (NCSU, 2014). 
The FAQ document was distributed through multiple 
channels, and there is some anecdotal evidence that it is 
making a difference as the response rates have risen back to 
the upper 40% range over the past few semesters. 
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